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Abstract — Given a time series of multicomponent measure- 
ments x{t), the usual objective of nonlinear blind source sepa- 
ration (BSS) is to find a "source" time series s{t), comprised of 
statistically independent combinations of the measured compo- 
nents. In this paper, the source time series is required to have 
a density function in (s, s) -space that is equal to the product of 
density functions of individual components. This formulation of 
the BSS problem has a solution that is unique, up to permutations 
and component-wise transformations. Separability is shown to 
impose constraints on certain locally invariant (scalar) functions 
of X, which are derived from local higher-order correlations of 
the data's velocity x. The data are separable if and only if they 
satisfy these constraints, and, if the constraints are satisfied, the 
sources can be explicitly constructed from the data. The method is 
illustrated by using it to separate two speech-like sounds recorded 
with a single microphone. 



I. Introduction 

Sensory devices often receive signals from multiple physical 
stimuli that evolve simultaneously but are unrelated to one 
another. In many of these situations, it is necessary to create 
separate representations of one or more of these stimuli by 
blindly processing the observed signals (i.e., by processing 
them without prior knowledge of the nature of the stimuli). In 
recent years, there has be considerable progress in the solution 
of this "blind source separation" (BSS) problem for the special 
case in which the signals and source variables are linearly 
related. However, although nonlinear BSS is often performed 
effortlessly by humans, computational methods for doing this 
are quite limited [1]. 

Consider a time series of data x{t), where x is a multiplet 
of N measurements (x^ for fc = l,2,...,iV). The usual 
objectives of nonlinear BSS are: 1) determine if these data are 
instantaneous mixtures of N statistically independent source 
components s{t) 

x{t) = f[s{t)l (1) 

where / is a possibly nonlinear, invertible A^-component 
mixing function; 2) if this is the case, compute the mixing 
function. In other words, the problem is to find a coordinate 
transformation /^^ that transforms the observed data x{t) 
from the measurement-defined coordinate system (x) on state 
space to a special source coordinate system (s) in which the 
components of the transformed data are statistically indepen- 
dent. Let ps{s) be the state space probability density function 
(PDF) in the source coordinate system, defined so that ps{s)ds 
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is the fraction of total time that the source trajectory s{t) is 
located within the volume element ds at location s. In the usual 
formulation of the BSS problem, the source components are 
required to be statistically independent in the sense that their 
state space PDF is the product of the density functions of the 
individual components 



N 



PS{S) = n PkiSk)- 



(2) 



fe=l 



In every formulation of BSS, multiple solutions can be created 
by permutations and component-wise transformations of any 
one solution. However, it is well known that the criterion in ^ 
is so weak that it suffers from a much worse non-uniqueness 
problem: namely, in this form of the BSS problem, multiple 
solutions can be created by transformations that mix the source 
variables (see [2] and references therein). 

The issue of non-uniqueness can be circumvented by con- 
sidering the data's trajectory in (s, s)-space (s — ds/dt) 
instead of s-space (i.e., state space). First, let ps{s,s) be the 
PDF in this space, defined so that ps{s, s)dsds is the fraction 
of total time that the location and velocity of the source 
trajectory are within the volume element dsds at location 
(s, s). An earlier paper [3] described a formulation of the BSS 
problem in which this PDF was required to be the product of 
the density functions of the individual components 



N 



Ps{s,s) = Y[Pk{sk,Sk)- 



(3) 



fe=l 



Separability in (s, s)-space is a stronger requirement than 
separability in state space. To see this, note that (|2|l can be 
recovered by integrating both sides of ^ over all velocities, 
but the latter equation cannot be deduced from the former one. 
In fact, it can be shown that (O is strong enough to guarantee 
that the BSS problem in (s, s)-space has a unique solution, up 
to permutations and component-wise transformations [3]. Fur- 
thermore, this type of statistical independence has the virtue 
of being satisfied by almost all classical physical systems that 
are composed of non-interacting subsystems, which are the 
generators of most signals of interest. 

The author previously demonstrated [3] that the (s, s)-space 
PDF of a time series induces a Riemannian geometry on the 
state space, with the metric equal to the local second-order 
correlation matrix of the data's velocity. Nonlinear BSS can be 
performed by computing this metric in the x coordinate system 
(i.e., by computing the second-order correlation of x at each 
point x), as well as its first and second derivatives with respect 
to X. However, although this is a mathematically correct and 
complete method of solving the nonlinear BSS problem, it 
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suffers from a practical difficulty: namely, if the dimensionality 
of state space is high, a great deal of data is required to 
cover it densely enough in order to calculate these derivatives 
accurately. The current paper [4] shows how to perform 
nonlinear BSS by computing higher-order local correlations 
of the data's velocity, instead of computing derivatives of 
its second-order correlation. This approach is advantageous 
because it requires much less data for an accurate computation. 
For example, in the synthetic speech separation experiment 
in Section III, the new method can separate two synthetic 
utterances recorded with a single microphone after minutes 
of observation, rather than the hours of observation required 
by the differential geometric method. 

The method described in this paper differs significantly from 
the methods proposed by other investigators because it uses a 
criterion of statistical independence in (s, s)-space, instead of 
state space. In addition, there are technical differences between 
the proposed method and conventional ones. First of all, the 
technique in this paper exploits statistical constraints on the 
data that are locally defined in state space, in contrast to 
the usual criteria for statistical independence that are global 
conditions on the data time series or its time derivatives [5]. 
Furthermore, unlike many other methods [6], [7], the mixing 
function is derived in a constructive, deterministic, and non- 
parametric manner, without employing iterative algorithms, 
without using probabilistic learning methods, and without 
parameterizing it with a neural network architecture or other 
means. In addition, the proposed method can handle any 
differentiable mixing function, unlike some other techniques 
that only apply to a restricted class of mixing functions [8]. 

The next section describes how to separate two-dimensional 
data into two one-dimensional source variables. Section III 
illustrates the method by using it to separate two simultaneous 
speech-like sounds that are recorded with a single microphone. 
The implications of this work are discussed in the last section. 
The appendix describes how the method can be generalized 
to separate data of arbitrary dimensionality into possibly 
multidimensional source variables. 

II. Method 

The BSS procedure, which is described in this section, is 
initiated by constructing scalar functions on the data space 
from combinations of local velocity correlations. The values of 
these scalars are invariant under any nonlinear transformations 
of coordinates on the data space. It is relatively easy to 
show that separability imposes necessary conditions on these 
scalar functions in the source coordinate system. Because of 
their scalarity, these conditions can readily be transferred to 
the measurement-defined coordinate system (x), where they 
can be tested with the data. If the data do not satisfy these 
necessary conditions, the data are simply not separable. If 
the data do satisfy these conditions, we show that there is 
only one possible source coordinate system, and it can be 
explicitly constructed. The data can then be transformed into 
this putative source coordinate system to see if their PDF 
and/or correlations factorize there. The data are separable if 
and only if this factorization occurs. 



The first step is to construct local correlations of the data's 
velocity, such as 



CkL.X^) = < i^k - ik){ii - ii) ■ 



(4) 



where x —< x >x, where the bracket denotes the time average 
over the trajectory's segments in a small neighborhood of 
X, where I < k, I < N, and where ". . ." denotes possible 
additional indices on the left side and corresponding factors 
of i — i on the right side. The definition of the PDF implies 
that this velocity correlation is one of its moments 

/ p{x, x){xk - Xk){xi - xi) .. .dx 



Cki...{x) 



J p{x, x)dx 



(5) 



where p{x, x) is the PDF in the x coordinate system. Inciden- 
tally, although (|5]) is useful in a formal sense, in practical ap- 
plications, all required correlation functions can be computed 
directly from local time averages of the data ((HJi), without 
explicitly computing the data's PDF. Also, note that velocity 
"correlations" with a single subscript vanish identically. 

Next, let M{x) be a local N x N matrix, and use it to define 
A/-transformed velocity correlations 



Iki...{x) 



E 

l<k'l'....<N 



Mkk'ix)Mu'ix)...Ck'i'...{x), (6) 



where ". . ." denotes possible additional indices of / and C, 
as well as corresponding factors of M{x). Because Cki{x) is 
positive definite at any point x, it is always possible to find 
an M{x) such that 

Iki{x) = Ski (7) 



l<m.<N 



Iklmm{x) = Dkl{x), 



(8) 



where D{x) is a diagonal NxN matrix. Such an M{x) can 
always be constructed from the product of three matrices: 1) 
a rotation that diagonalizes Cki{x), 2) a diagonal rescaling 
matrix that transforms this diagonalized correlation into the 
identity matrix, 3) another rotation that diagonalizes 



E 

l<m<JV 



Cklmrn (•^) ; 



after the fourth-order correlation has been transformed by the 
first rotation and the rescaling matrix. As long as the last- 
diagonalized matrix is not degenerate, M{x) is unique, up to 
arbitrary local permutations and reflections. In almost all real- 
istic applications, the velocity correlations will be continuous 
functions of the state space coordinate x. Therefore, in any 
neighborhood of state space, there will always be a continuous 
solution for M{x), and this solution is unique, up to arbitrary 
global reflections and permutations. 

In order to show that the A /-transformed velocity corre- 
lations (i.e., the Ikim...{x)) transform like scalars, imagine 
constructing these quantities in some other coordinate system 
x' . An M-matrix that satisfies © and ^ in the x' coordinate 
system is given by 



l<in<N ' 



(9) 
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where M is a matrix that satisfies O and ([8]) in the x coor- 
dinate system. To prove this, substitute this equation into the 
definition of I'^^ {x'). Because velocity correlations transform 
as contravariant tensors, the partial derivative factors within 
M' transform correlations from the x' coordinate system to 
the X coordinate system, leading to 

I'uA^') = E Ml,,{x')KW)...C',,,„{x') 

l<k'J',...<N 

E Mkk'{x)Mw{x)...Ck'i'...{x) 

l<k',l',...<N 

= Iki...{x). 

Therefore, because Iki{x) and Ikimn{x) satisfy (|7]i and (O, 
so do I'kiix') and I'kimn^^')^ thereby proving that (|9]l is one 
of the solutions for M'{x') in the x' coordinate system. All 
other solutions for M'(x') differ from this one by global 
reflections and permutations. Similar reasoning shows that, for 
any choice of M' and M, each of the functions {x') 
equals the corresponding function Iki...{^)^ 1° possible 
global permutations and reflections. In other words. 



4...(^') = 



E 

l<k'l'...<N 



Pkk'Pw ■ ■ ■ Ik'v...{x), 



(10) 



where Pfe^/ denotes an element of a product of permutation, 
reflection, and identity matrices. In other words, the functions 
Iki...{^) transform as scalar functions on the state space, 
except for possible reflections and index permutations. 

We now assume that the system is separable and derive 
some necessary conditions on these scalar functions in the 
source coordinate system (s). Because these separability con- 
ditions involve scalar functions, they can then be transferred 
to the measurement-defined coordinate system ix), where they 
can be tested with the data. In order to make the notation 
simple, it is assumed that = 2 in the following. However, 
the appendix describes how the methodology can be general- 
ized in order to separate higher-dimensional data into possibly 
multidimensional source variables. 

Separability implies that there is a transformation /^^ from 
the X coordinate system to a source coordinate system (s) in 
which ^ is true. Because of (|5]l, the velocity correlation func- 
tions in the s coordinate system are products of correlations 
of the independent sources 



C. 



SI. 



(11) 



where 1 . . . and 2 . . . denote arbitrary numbers of indices equal 
to 1 and 2, respectively. It follows from this equation and from 
the vanishing of all velocity "correlations" with one index that 
the source variable correlations Cski{s) and 



E dsklmmix) 



l<m<N 



are diagonal. Therefore, in the s coordinate system, (|7]l and 
(|8]l are satisfied by a diagonal matrix AIs{s) of the form 



Ms{s) = 



f Msiisi) 



(12) 



V A/S2(S2), 

It follows from (fTTT i and ( fT2] i that the scalar functions Iski...{^) 
with all subscripts kl . . . equal to 1 (2) must equal the 



corresponding functions derived for subsystem 1 (2), and 
these latter functions depend on si (s2) alone. Although these 
constraints were derived in the s coordinate system, scalarity 
(((Toll) implies that these separability conditions are true in 
all coordinate systems, except for possible permutations and 
reflections. Therefore, in the measurement-defined coordinate 
system (x), the functions Iki...{^) '^i^h all subscripts equal 
to 1 must be functions of either si{x) or S2{x). Likewise, 
the functions Iki...{^) with all subscripts equal to 2 must 
be functions of the other source variable is2{x) or si{x), 
respectively). 

This coordinate-system-independent consequence of separa- 
bility can be used to perform nonlinear BSS in the following 
manner: 

1) Use (|4|i to compute velocity correlations Cki...{x) from 
the data x{t). 

2) Use linear algebra to find a continuous matrix M{x) that 
satisfies (|7]i and (O. 

3) Use ^ to compute the functions Iki...{x). 

4) Plot the values of the triplets 

Ia{x) = {hiiix), /iiiii(a:)} (13) 

Ib{x) = {l222{x), l2222{x), /22222(a;)} (14) 

as X varies over the measurement-defined coordinate 
system. 

5) If the plotted values of Ia and/or Ib do not lie in 
one-dimensional subspaces within the three-dimensional 
space of the plots, Ia{x) and/or Ib{x) cannot be func- 
tions of single source components (si(x) or 52(2;)) as 
required by separability, and the data are not separable. 

6) If the plotted values of both I a and Ib do lie on one- 
dimensional manifolds, define one-dimensional coordi- 
nates {ga and ctb, respectively) on those subspaces. 
Then, compute the function a{x) ~ ((7^(2;), ctb (.t)) 
that maps each coordinate x onto the value of a, which 
parameterizes the point {Ia{x),Ib{x))- Notice that, 
because of the Takens' embedding theorem [9], x is 
invertibly related to the six components of Ia{x) and 
Ib{x), and, therefore, it is invertibly related to a. 

7) Transform the PDF (or correlations) of the measure- 
ments from the x coordinate system to the a coordinate 
system. The data are separable if and only if the PDF 
factorizes (the correlations factorize) in the a coordinate 
system. 

The last statement can be understood in the following 
manner As shown above, separability implies that I a {x) must 
be a function of a single source variable (51(2;) or 52(2^)), and 
the Takens theorem implies that this function is invertible. 
Because Ia is also an invertible function of cr^, it follows that 
aA must be invertibly related to one of the source variables, 
and, in a similar manner, (Tb must be invertibly related to the 
other source variable. Thus, separability implies that <ta and 
aB are themselves source variables. It follows that the data are 
separable if and only if the PDF factorizes in the cr coordinate 
system. 

Although the above-described procedure will perform BSS 
for any mixing function, it is interesting to consider the special 
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case in which source variables exist that are linearly related 
to the measurements; namely, 

l</<2 

where is a constant 2x2 matrix. In general, the above BSS 
procedure will construct source variables cr that are related to 
these "linear" source variables by 

C7i{x) = gi{si{x)) = gi{WiiXi + W12X2) (16) 
(^2{x) = 32(52(2;)) = 32(^^21X1 + ^^222:2), (17) 

where gi and 32 are some invertible nonlinear transformations 
determined by the choice of the (Ti and CT2 coordinates, 
respectively. Therefore, at each point x the partial derivatives 

d(7i/dxk 
d(j2/dxu 

will be proportional to constant (i.e., x-independent) vectors 
(denoted by Uik and U2k), which are themselves proportional 
to the first and second rows of W, respectively. Furthermore, 
these vectors can be used to construct other linearly-related 
source variables 

Sk= ^kixi (18) 

1<1<2 

that are just rescaled versions of the ones in (flST l. Conse- 
quently, given the source variables a{x) produced by the BSS 
procedure, the following process can be used to determine 
whether these can be transformed into source variables that 
are linearly related to the measurements: 1) compute the 
above-mentioned partial derivatives and determine if each 
is proportional to an a;-independent vector; 2) if the partial 
derivatives do not satisfy this condition, there are no linearly- 
related source variables; 3) if the partial derivative do satisfy 
condition 1, transform the data into the s coordinate system 
in order to see if the data's PDF factorizes there. There are 
linearly-related source variables if and only if this factorization 
occurs. 

III. Numerical Example: Separating Two 
Speech-Like Sounds Recorded with a Single 
Microphone 

This section describes a numerical experiment in which two 
speech-like sounds were synthesized and then summed, as if 
they were simultaneously recorded with a single microphone. 
Each sound simulated an "utterance" of a vocal tract resem- 
bling a human vocal tract, except that: 1) it had one degree 
of freedom, instead of the 3-5 degrees of freedom of the 
human vocal tract; 2) its impulse response was characterized 
by one pole pair, instead of the 4-6 pole pairs characteristic of 
the human vocal tract. The methodology of Section II was 
blindly applied to a time series of two features extracted 
from the synthetic recording, in order to recover the time 
dependence of the state variable of each vocal tract (up to an 
unknown transformation on each voice's state space). BSS was 
performed with only 16 minutes of data, instead of the hours 
of data required to separate similar sounds using a differential 
geometric method [3]. 



Each speaker was simulated by having a simulated glottis 
drive a simulated resonant cavity that represented the vocal 
tract. The glottal waveform of the each "voice" was a series 
of spikes separated by a pitch interval (100 Hz and 160 Hz). 
The impulse response of each "vocal tract" was taken to be 
a characteristic damped sinusoid, whose amplitude, resonant 
frequency, and damping were linear functions of a single state 
variable. For each voice, a 16 minute utterance was produced 
by convolving its glottal waveform with the impulse response 
of its vocal tract, which was a function of a slowly-varying 
state variable. The state variable time series of each voice 
was synthesized by smoothly interpolating among successive 
states. The latter were chosen at 100-120 msec intervals so 
that the state variable time series of the two voices were 
statistically independent of each other The resulting utterances 
had energies differing by 2.4 dB, and they were summed and 
sampled at 16 kHz with 16-bit depth. Then, this "recorded" 
waveform was pre-emphasized and subjected to a short-term 
Fourier transform (using frames with 25 msec length and 5 
msec spacing). The log energies of a bank of 12 mel-frequency 
filters between 0-8000 Hz were computed for each frame, and 
these were then averaged over pairs of consecutive frames. 
These log filterbank outputs were nonlinear functions of the 
two vocal tract state variables. 

In order to blindly analyze these data, we first determined 
if any data components were redundant in the sense that they 
were simply functions of other components. Fig. [T^ shows the 
first three principal components of the log filterbank outputs 
during a typical short recording of the simultaneous utterances. 
Inspection showed that these data lay on a curved two- 
dimensional surface within the ambient 12-D space, making 
it apparent that they were produced by a "hidden" dynamical 
system with two degrees of freedom. The redundant compo- 
nents were eliminated by using dimensional reduction (princi- 
pal components analysis in small overlapping neighborhoods 
of the data) to establish a coordinate system x on this surface 
and to find x{t), the trajectory of the recorded sound in that 
coordinate system. Next, the BSS procedure in Section II 
was used to determine if x{t) was a nonlinear mixture of 
two source variables that were statistically independent of one 
another Following steps 1-4 of the BSS procedure, x{t) of 
the entire recording was used to compute invariants Iki...{x) 
with up to five indices, and the related functions Ia{x) and 
Ib{x) were plotted, as illustrated in Figs.[TJ)-c. It was evident 
that the plotted values of both Ia and Ib lay in or close 
to one-dimensional subspaces. Following step 6 of the BSS 
procedure, a dimensional reduction procedure [12] was used 
to define coordinates ((ta and as) on these one-dimensional 
manifolds, and a{x) = {aA{x),aB{x)) was computed. If the 
data were separable, a must be a set of source variables, and 
<7[x(t)] must describe the evolution of the underlying vocal 
tract states (up to invertible component-wise transformations). 
As illustrated in Figs. |2^-b and Figs. [3^-b, the time courses 
of the putative source variables (aA[x{t)],(TB[x{t)]) did re- 
semble distorted versions of the state variable time series 
that were originally used to generate the voices' utterances. 
The scatter plots in Fig. |2}; and Fig. ^ show that, in each 
case, the recovered source variable and the corresponding 
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(a) (b) (c) 

Fig. 1. (a) The first tliree principal components of log filterbank outputs of a typical short recording of two simultaneous speech-like sounds, (b) The 
distribution of the values of Ia{^) ( USt ). as x varied over the approximately two-dimensional manifold in (a), (c) The distribution of the values of Ib{^) 
( il4\ ). as X varied over the approximately two-dimensional manifold in (a). 
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(a) 



(b) 



(c) 



Fig. 2. (a) The time dependence of one of the source variables, blindly computed from a typical five-second segment of the data's trajectory x{t), by finding 
the coordinates of Iyi[x{t)] on the one-dimensional manifold in Fig. lb. (b) The state variable time series originally used to generate one of the speech-like 
sounds during the five-second recording analyzed in (a), (c) The scatter plot of the pairs of source and state variable values in (a) and (b). 






(a) 



(b) 



(c) 



Fig. 3. (a) The time dependence of one of the source variables, blindly computed from a typical five-second segment of the data's trajectory x{t), by finding 
the coordinates of Ig[x{t)] on the one-dimensional manifold in Fig. Ic. (b) The state variable time series originally used to generate one of the speech-like 
sounds during the five-second recording analyzed in (a), (c) The scatter plot of the pairs of source and state variable values in (a) and (b). 



State variable were related by a nonlinear transformation that 
was nearly monotonic, except for the effects of noise due 
to the limited number of data samples. Thus, starting with 
a single-microphone recording, the BSS procedure was able 
to extract the information encoded in the time series of each 
speaker's state variable. The time course of the analogous mul- 
tidimensional state variable of the human vocal tract contains 
the speech content of each utterance. This indicates that the 
BSS procedure is capable of recovering the speech content 
of superposed utterances, without recovering their original 



waveforms. 



IV. Discussion 

In a previous paper [3], the nonlinear BSS problem was 
reformulated in (state, state velocity)-space, instead of state 
space as in conventional formulations. This approach is attrac- 
tive because: 1) the reformulated BSS problem has a unique 
solution in the following sense: either the data are insepa- 
rable, or they can be separated by a mixing function that is 
unique (up to permutations and transformations of independent 
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source variables); 2) statistical independence in (state, state 
velocity)-space is manifested by almost all classical physical 
systems that are composed of non-interacting subsystems. This 
paper [4] shows how a general solution of this problem can 
be constructed in a deterministic manner, which avoids the 
difficulties of the iterative, probabilistic, and parametric BSS 
techniques proposed by other investigators. Furthermore, an 
accurate computation can be performed with far less data than 
that required by the differential geometric solution, previously 
proposed by the author [3]. 

The BSS procedure in Section II shows how to compute 
{aA[x{t)], aB[x{t)]), the trajectory of each independent sub- 
system in a specific coordinate system on that subsystem's 
state space. In many practical applications, a pattern recog- 
nition "engine" has been trained to recognize the meaning of 
trajectories of one subsystem (e.g., "A") in another coordinate 
system (e.g., sa) on that subsystem's state space. In order 
to use this information, it is necessary to know the trans- 
formation to this particular coordinate system {(ta sa)- 
For example, subsystem A may be the vocal tract of speaker 
A, and subsystem B may be a noise generator of some sort. 
In this example, we may have trained an automatic speech 
recognition (ASR) engine on the quiet speech of speaker A 
(or, equivalently, on the quiet speech of another speaker who 
mimics A in the sense that their state space trajectories are 
related by an invertible transformation when they speak the 
same utterances). In order to recognize the speaker's utterances 
in the presence of B, we must know the transformation from 
the vocal tract coordinates recovered by BSS (cta) to the 
coordinates used to train the ASR engine (sa)- This mapping 
can be determined by using the training data to compute more 
than 2d A invariants (like those in (|6]l) as functions of sa- 
These must equal the invariants of one of the subsystems 
identified by the BSS procedure, up to a global permutation 
and/or reflection ((fToli). This global transformation can be 
determined by permuting and reflecting the distribution of 
invariants produced by the training data, until it matches the 
distribution of invariants of one of the subsystems produced 
by the BSS procedure. Then, the mapping (ja ^ sa can 
be determined by finding paired values of cta and sa that 
correspond to the same invariant values within these matching 
distributions. This type of analysis of human speech data is 
currently underway. 

Appendix 

Separating data of any dimensionality 
into possibly multidimensional sources 

The procedure in Section II is capable of separating two- 
dimensional data into one-dimensional source variables. This 
appendix describes the solution of the more general nonlinear 
BSS problem in which data of any dimensionality may be 
separated into possibly multidimensional source variables, 
each of which is statistically independent of the others but 
each of which may contain statistically dependent components. 
This is sometimes called multidimensional independent com- 
ponent analysis, subspace independent component analysis, or 
independent subspace analysis [10], [11]. 



Separability implies that there is a transformation /^^ from 
the X coordinate system to a source coordinate system (s = 
{sa,sb)) in which 

ps(s,s) = PAisA,SA)pBisB,SB), (19) 

where sa is a possibly multidimensional source variable with 
dA components and is a possibly multidimensional source 
variable with = N — dA components. Because of (|5]l, the 
velocity correlation functions in the s coordinate system are 
products of correlations of independent sources 

Csa...b..Xs) = Csa...{sA)Csb...{sB), (20) 

where a . . . and & . . . denote arbitrary series of indices in the 
ranges 1 < a < d^i and dA + I < b < N, respectively. 
It follows from this equation and from the vanishing of all 
velocity "correlations" with one index that the source variable 
correlations Cski{s) and 

Csklmm{x) 

l<m<N 

have block-diagonal forms with dAxdA and dBxds upper 
and lower blocks, respectively. Consequently, in the s coor- 
dinate system, (|7]i and (O are satisfied by a block-diagonal 
matrix Ms{s) of the form 

/ Msa{sa) a 
^''^^■'^={ Msb{sb))^ ^^'^ 

where Ms a and Msb are matrices that satisfy dTji and (O 
for the A and B subsystems, respectively. In order to prove 
that ( l2Tl i satisfies substitute it into the definition of Iski, 
and note that each block of Ms is defined to transform the 
corresponding block of Csu into an identity matrix. In order 
to prove that (l2Tl l satisfies (H), substitute it into the definition 
of 

iski (22) 

l<m<Af 

Then, note that: 1) when k and I belong to different blocks, 
each term in this sum vanishes because it factorizes into a 
product of a one-index correlation and a three-index corre- 
lation; 2) when k and I belong to the same block and are 
unequal, each term with m in the other block contains a factor 
equal to Iski, which vanishes, as proved above; 3) when k and 
I belong to the same block and are unequal, the sum over m in 
the same block vanishes, because each block of Ms is defined 
to satisfy (O for the corresponding subsystem. 

It follows from (|6]l, (|20] |. and (ISTT i that the scalar functions 
Iski...{^) 'with all subscripts fcL . . in the range \ < k^ I < dA 
(or in the range d^i + 1 < k, I < N) must depend on sa 
(or sb) alone. Although these constraints were derived in 
the s coordinate system, scalarity (dToli) implies that these 
separability conditions are true in all coordinate systems, ex- 
cept for possible permutations. Therefore, in the measurement- 
defined coordinate system (x), it must be possible to partition 
the indices of x {k — 1,2,...,N) into A and B groups 
(containing dA indices and "_B" indices, respectively) 
so that the functions Iki...{^) with all subscripts in the A (or 
B) group are functions of sa{x) (or sb{x)) alone. 
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This coordinate-system-independent consequence of separa- 
bihty can be used to perform nonlinear BSS in the following 
manner: 

1) Use (lU to compute velocity correlations Cki...{x) from 
the data x{t). 

2) Use linear algebra to find a continuous matrix M{x) that 
satisfies (|7]i and (HJ. 

3) Use (|6]l to compute the functions Iki...{x). 

4) Consider each choice of an integer cIa in the range 
1 < (Ia < N , and consider each way of partitioning 
the data indices (fc = l,2,...,Af) into A and B groups 
(containing "A" indices and = N — dA "B" 
indices, respectively). For each of these choices, let 
Ia{x) (Ib{x)) be any set of more than 2dA (2^5) of the 
functions Iki...{x) for which all subscripts belong to the 
A (B) group, and plot the values of Ia{x) and Ib{x) 
as X varies over the measurement-defined coordinate 
system. 

5) Suppose that, for all of the choices in step 4, 
the plotted values of Ia and/or Ib do not lie in 

-dimensional (ds -dimensional) subspaces within the 
higher-dimensional space of the plots. Then, there is 
no way that Ia{x) and Ib{x) can be functions of 
single source variables {sa{x) and sb{x)) as required 
by separability, and the data are not separable. 

6) Suppose that, for one or more of the choices in 
step 4, the plotted values of both Ia and Ib 
do lie in d^i -dimensional and -dimensional mani- 
folds, respectively. In that case, define -dimensional 
(da -dimensional) coordinates aA (o'b) on those 
subspaces. Then, compute the function <j{x) = 
{aA{x),aB{x)) that maps each coordinate x onto 
the value of cr, which parameterizes the point 
{Ia{x), Ib{x))- Notice that, because of the Takens' 
embedding theorem [9], x is invertibly related to the 
2A^ + 2 (or more) components of Ia{x) and Ib{x), and, 
therefore, it is invertibly related to cr. 

7) Transform the PDF (or correlations) of the measure- 
ments from the x coordinate system to the a coordinate 
system. The data are separable, and a a and (Tb are 
source variables if and only if the PDF factorizes (the 
correlations factorize) in a cr coordinate system created 
in this way. 

The last statement can be understood in the following 
manner As shown above, separability implies that, for some 
choice of dA and index partitioning, I a {x) must be a function 
of sa{x), and the Takens theorem implies that this function 
is invertible. Because Ia is also an invertible function of 
(7A, it follows that aA must be invertibly related to sa, and, 
in a similar manner, cr^ must be invertibly related to sb- 
Thus, separability implies that cr^ and ctb are themselves 
source variables and, therefore, the PDF factorizes in the a 
coordinate system. Finally, note that, if the data are separable, 
the same procedure can then be used to determine if each 
multicomponent source variable (aA or cr^) can be further 
separated into lower-dimensional source variables. 

The above-described procedure will perform BSS for any 



linear or nonlinear mixing. However, a few comments should 
be made about the special case in which source variables exist 
that are linearly related to the measurements; namely, 

SAa = ^ WAakXk (23) 
l<fc<Af 

SBb = ^ WBbkXk, (24) 
l<fc<Af 

where 1 < a < dA, where 1 < b < dB, and where Wa and 
Wb are constant dA xN and dB xN matrices, respectively. 
In general, the above BSS procedure will construct source 
variables cr that are related to these "linear" source variables 
by 

o-A = 9a{sa) (25) 
<7B = 9b{sb), (26) 

where gA and qb are some nonlinear transformations (with dA 
and dB components, respectively), determined by the choice 
of the OA and cr^ coordinates, respectively. Therefore, at each 
point X, the sets of partial derivatives 

d<7Aa/dxk for 1 < a < dA 
daBb/dxk for 1 < 6 < 

will be lie in the subspace spanned by the rows of Wa and 
Wb, respectively. Let UAak and UBbk (for 1 < a < d^i 
and 1 < 6 < ds) denote any sets of constant vectors that 
span these two subspaces. Then, another set of linearly-related 
source variables is given by 

SAa = ^ UAakXk (27) 
l<fc<A' 

SBb = ^ UBbkXk, (28) 

i<fc<Ar 

which are just linear combinations of the ones in ( l23T l and (l24l i. 
respectively. Consequently, given the source variables cr(a;) 
produced by the BSS procedure, the following process can 
be used to determine whether these can be transformed into 
source variables that are linearly related to the measurements: 
1) compute the above-mentioned sets of partial derivatives and 
determine if each set is spanned by the appropriate number 
of x-independent vectors; 2) if one or both sets of partial 
derivatives do not satisfy this condition, there are no linearly- 
related source variables; 3) if both sets of partial derivative do 
satisfy condition 1, transform the data into the s coordinate 
system in order to see if the data's PDF factorizes there. 
There are linearly-related source variables if and only if this 
factorization occurs. 

References 

[1] C. Jutten and J. Karhunen, "Advances in nonlinear blind source sep- 
aration," in Proceedings of the 4*'' International Symposium on Inde- 
pendent Component Analysis and Blind Signal Separation, Nara, Japan, 
April 2003. 

[2] A. Hyvarinen and P. Pajunen, "Nonlinear independent component anal- 
ysis: existence and uniqueness results," Neural Networks, vol. 12, pp. 
429-439, 1999. 

[3] D. N. Levin, "Using state space differential geometry for nonlinear blind 
source separation," / Applied Physics, vol. 103, art. no. 044906, 2008. 



8 



[4] D. N. Levin, "Using signal invariants to perform nonlinear blind source 
separation," in Proc. of the 8"' International Conference on Independent 
Component Analysis and Signal Separation, LNCS, vol. 5441, T. Adali, 
C. Jutten, J. M. T. Romano, A. K. Barros (eds). Heidelberg: Springer, 
2009, pp. 58-65. 

[5] S. Lagrange, L. Jaulin, V. Vigneron, C. Jutten, "Analytic solution of 
the blind source separation problem using derivatives," in Independent 
Component Analysis and Blind Signal Separation, LNCS, vol. 3195, 
C. G. Puntonet and A. G. Prieto (eds). Heidelberg: Springer, 2004, pp. 
81-88. 

[6] H. H. Yang, S I. Amari, A. Cichocki, "Information-theoretic approach 

to blind separation of sources in non-linear mixture," Signal Processing, 

vol. 64, pp. 291-300, 1998. 
[7] S. Haykin, Neural Networks - A Comprehensive Foundation. New York: 

Prentice Hall, 1998. 
[8] A. Taleb and C. Jutten, "Source separation in post-nonlinear mixtures," 

IEEE Trans. Signal Process., vol. 47, pp. 2807-2820, 1999. 
[9] T. Sauer, J. A. Yorke, M. Casdagli, "Embedology," / Statistical Physics, 

vol. 65, pp. 579-616, 1991. 
[10] J.-F. Cardoso, "Multidimensional independent component analysis," in 

Proc. IEEE International Conference on Acoustics, Speech, and Signal 

Processing, vol. 4, Seattle, 12-15 May, 1998, pp. 1941-1944. 
[11] Y. Nishimori, S. Akaho, M. D. Plumbley, "Riemannian optimization 

method on the flag manifold for independent subspace analysis," in Proc. 

6*'' International Conference on Independent Component Analysis and 

Blind Source Separation, LNCS, vol. 3889. Berlin: Springer, 2006, pp. 

295-302 

[12] S. T. Roweis and L. K.Saul, "Nonlinear dimensionality reduction by 
locally Unear embedding," Science, vol. 290, pp. 2323-2326, 2000. 



